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ABSTRACT 

Cal  cul  ati  ons  of  sonar-system  performance  for  anti-submari  ne  warfareyi  el  d  detecti  on  probabi  I  ity 
as  a  function  of  range  to  the  target;  whereas  sonar  operators  typically  think  in  terms  of  a 
'detection  range'.  This  Note  considers  how  to  connect  these  two  concepts  for  active  sonar.  Four 
types  of  probability  are  explored  and  their  relative  advantages  teased  out  in  the  light  of  real- 
world  examples.  Itisconduded  that  cumulative  probabi  I  ity  of  track  initiation  provides  the  most 
practical  route  to  a  definition  of  detection  range,  for  two  reasons.  First,  unlike  detection 
probability  as  usually  computed,  it  produces  values  that  operators  are  likely  to  regard  as 
bel  i  evabl  e.  Second  I  y ,  i  t  ref  I  ects  more  cl  osel  y  than  the  stand  ard  d  etecti  on  probabi  I  ity  the  steps  that 
operators  typically  go  through  in  deciding  to  declare  a  detection. 
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CumulativeTrack-lnitiation  Probability  as  a  Basisfor 
Assessing  Sonar- System  Performance  in 
A  nti  -Su  bmari  ne  W  arfare 


Executi  ve  Su  mmary 

The  key  quantity  that  an  operator  wishes  to  know  about  any  surveillance  asset  is  its 
detection  range— how  far  away  a  target  of  interest  can  bedetected.  Sonar  systems  for  anti¬ 
submarine  warfare  are  no  exception;  performance  is  often  specified  in  terms  of  an 
achievable  detection  range.  However  the  concept  of  a  fixed  'rangeof  theday'  does  not  sit 
easily  with  the  realities  of  highly  variable  acoustic  propagation  and  ambient  noise  in  the 
sea.  Real-world  variability  and  fluctuations  are  more  readily  expressed  in  terms  of 
detecti  on  probabi  I  ity  as  a  functi  on  of  target  range  and  depth.  Thi  s  report  exami  nes  ways  of 
deriving  detection  range  from  detection  probability  for  active  sonar  through  a 
consideration  of  four  types  of  probability: 

•  single-ping  detection  probability  pd,  which  is  the  result  of  acoustic-propagation 
calculations, 

•  cumulative  detection  probability  Pd,  which  is  the  probability  of  obtaining  a 
detection  on  the  current  ping  or  from  any  previous  ping  since  the  start  of  the 
scenario  vignette, 

•  'local'  track-initiation  probability  pti,  which  is  the  probability  of  starting  a  track 
after  5  consecuti  ve  pi  ngs,  assu  mi  ng  that  thetrack-i  ni  ti  ati  on  rul  e  requ  i  res  3  or  more 
detections  in  the  5  pings,  and 

•  cumulative  track-initiation  probability  Pti,  which  is  the  probabi  I  ity  that  a  track  is 
started  on  the  current  ping,  or  was  started  on  any  previous  ping. 

Our  analysis  concludes  that  Pti  provides  the  best  basis  for  defining  detection  range.  The 
advantages  and  disadvantages  of  Pti  compared  with  the  other  probabilities  are 
summarised  in  the  traffic-light  diagram  on  the  next  page.  In  detail: 

•  C  u  mu  I  ati  ve  probabi  I  i  ties,  whether  Pti  orPd,  requireascenariofortheircalculation. 
H  ence  thei  r  val  ues  are  more  scenari  o  dependent  than  pd  or  pti . 

•  Cumulative  probabilities  provide  a  basis  for  comparing  sonar  systems  with 
different  ping  rates  or  transmission  modes,  unlikepd  orpti. 

•  Single-ping  pd,  by  definition,  does  not  allow  the  analysis  of  the  effect  of  a  long 
series  of  pings,  unlikethe  cumulative  probabilities.  Local  pti  considers  groups  of 
5pingsonly  (or  n  pings  where  the  track-initiation  ruleism-in-n). 

•  Track-initiation  probabilities  provide  some  recognition  of  the  problem  of  false 
d etecti  ons,  u n I  i  ke  d etecti  on  probabi  I  i  ti  es,  w  h i  I  e  ski  rti  ng  arou nd  the  d  i ff  i  cu  1 1  i  ssu e 
of  data  association. 

•  As  regards  several  sonar  systems  acting  together  and  pooling  data,  pd  providesno 
basis  for  analysis.  Combinations  of  several  pd  values  are  required,  as  is  the 


si tuati  on  the  other  three  probabi  I  iti  es.  H  owever,  P  d  permits  the  analysi  s  of  track- 
I  evel  data  fusi  on  onl  y ,  w  hereas  pti  and  P  ti  al  so  al  I  ow  the  anal  ysi  s  of  detecti  on-l  evel 
fusion. 

•  Si  ncecumulati  veprobabi  I  ities  never  decreasewith  time,  they  providean  unambig¬ 
uous  basis  for  defining  detection  range,  unlikepd  orpti. 

•  The  actual  definition  of  detection  range,  however,  requires  the  choice  of  a 
probability  value,  which  is  unavoidably  arbitrary.  All  probabilities  have  this 
deficiency,  but  it  is  worst  for  cumulative  detection  probability  Pd,  because  the 
'natural'  choice  of  50%  gives  unrealistically  large  detection  ranges  in  most 
environments.  On  the  other  hand,  experience  suggests  that  Pti  =50%  works  well, 
at  least  for  persistent  sonar  systems  like  ship-borne  hull-mounted  sonars  and 
towed  arrays. 

The  purpose  of  any  pi  ece  of  mi  I  i  tary  operati  ons  anal  ysi  s  i  s  to  i  nform  a  deci  si  on  maker.  The 
success  of  the  analysis  is  closely  related  to  how  useful  the  client  finds  it.  Analytical 
techniques  based  on  cumulative  track-initiation  probability  have  been  used  to  inform 
decisions  in  support  of  a  major  defence  acquisition  project,  to  the  satisfaction  of  the  client. 


Summary  of  strengths  and  weaknesses  of  various  probabilities  for  the  purposes  of  defining  the  'detection 
range'  of  a  sonar  system. 


Probability: 

single-ping 

detection 

probability 

local 

track-initiation 

probability 

cumulative 

detection 

probability 

cumulative 

track-initiation 

probability 

independent  of  scenario* 

• 

• 

• 

• 

allows  comparison  of  sonars  with 
different  ping  rates  or  modes 

• 

• 

• 

• 

provides  some  recognition  of  the 
i nfl uence  of  false  detections 

• 

• 

• 

• 

i  ncl  udes  effect  of  many  repeated 
pings 

• 

• 

5  pings  only 

• 

• 

provides  a  basis  for  analysing 
networks  of  sonar  systems 

• 

• 

• 

track- level 
fusion  only 

• 

unambiguous  definition  of  detection 
range 

• 

• 

• 

• 

no  arbitrariness  in  definition 
of  detection  range 

• 

pd  =50% 
acceptable 

• 

pti  =50% 
acceptable 

• 

Pd  =50% 
unrealistic 

• 

Ptj  =50% 
acceptable 

*rhis  refers  to  the  disposition  and  motion  of  the  platforms  in  the  scenario.  All  probabilities  are  absolutely 
dependent  on  the  environment,  time  of  year,  nature  of  the  target  and  target  depth. 
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1.  Introduction 


The  key  quantity  that  an  operator  wishes  to  know  about  any  surveillance  asset  is  its  detection 
range  — how  far  away  a  target  of  interest  can  be  detected.  Sonar  systems  for  anti-submarine 
warfare  (ASW)  are  no  exception;  performance  is  often  specified  in  terms  of  an  achievable 
detection  range.  Unfortunately  this  leads  to  difficulties  in  the  case  of  sonar;  for  the  concept  of 
a  fixed  'range  of  the  day'  does  not  sit  easily  with  the  realities  of  highly  variable  acoustic 
propagation  and  ambient  noise  in  the  sea  [1],  for  reasons  that  are  illustrated  in  §2. 

This  note  advocates  the  use  of  'cumulative  probability  of  track  initiation'  Pti  as  a  route  to  the 
computation  of  realistic  and  operator-believable  active  sonar  detection  ranges  from  the  results 
of  acoustic-propagation  calculations.  The  concept  is  not  new— Pti  has  been  used  for  ASW 
analysis  at  least  once  before  [2]  —  but  the  advantages  of  its  use  are  not  as  widely  recognised  as 
they  deserve  to  be. 

Cumulative  track-initiation  probability  is  defined  in  §2.2.1,  where  an  algorithm  for  its 
calculation  is  described  and  its  advantages  are  canvassed.  Techniques  similar  to  those 
described  in  §2.2.1  were  used  to  inform  decisions  in  support  of  a  major  defence  acquisition,  to 
the  satisfaction  of  the  client. 


2.  Connecting  Detection  Probability  and 
Detection  Range 

2.1  Single-Ping  Detection  Probability 

Calculations  of  the  performance  of  ASW  sonar  systems  typically  give  results  as  single-ping 
detection  probability  ptp  This  is  the  probability  that  the  system,  after  pinging  once,  will 
register  an  echo  from  the  submarine  as  a  detection.  State-of-the-art  sonar-system  performance 
calculations  are  very  complicated,  taking  into  account  many  details  of  underwater  acoustic 
propagation,  the  ambient  noise  field  in  the  sea  and  the  operation  of  the  acoustic  processing 
system.  Figure  1(a)  shows  a  typical  result:  p<q  is  colour-coded  as  a  function  of  the  target 
submarine's  range  and  depth.(a)  Figure  1(b)  shows  part  of  the  data  in  (a)  in  a  more 
conventional  way,  as  a  plot  of  Pj  versus  range  for  a  particular  submarine  depth,  in  this  case 
50  m. 


(a)  Detection  probability  may  also  be  a  function  of  target  bearing  and  aspect  angle;  these  refinements  are 
omitted  here,  but  their  inclusion  causes  no  insurmountable  complication  for  the  main  point  of  this 
Note,  the  calculation  of  Pti . 
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Target  range  (km) 

Figurel:  Example  of  results  of  sonar-system  performancecalculations  for  a  notional  realistic  sonar  system, 
geographic  location,  time  of  year  and  target:  (a)  colour-coded  contour  plot  of  single-ping  detection 
probability  pd  as  a  function  of  target  range  and  depth  and  (b)  the  same  as  in  (a),  but  a  conventional 
plot  of  pd  against  target  range  for  a  particular  target  depth  (50  m  in  this  case).  T  hegrey  region  at  the 
bottom  of  pan  el  (a)  shows  the  profile  of  the  sea  floor.  N  ote  the  very  different  length  scales  on  the  two 
axes  in  this  panel. 


It  should  be  carefully  noted  that  the  term  'detection'  is  used  here  in  an  engineering  sense:  it 
means  no  more  than  that  the  target  of  interest  has  returned  sufficient  acoustic  energy  to  cause 
the  sonar  system  to  display  a  spot  on  a  screen.  This  is  often  rather  less  than  what  operators 
mean  by  'detection',  which  includes  elements  of  identification  and  the  elimination  of  false 
sonar  returns. 


Sometimes,  a  detection  range  is  derived  from  a  plot  like  Figure  1(b)  by  picking  a  pd  value,  say 

50%,  and  adopting  the  range  at  which  pd  has  that  value.  Several  criticisms  can  be  levelled  at 

this  method: 

1.  The  pd  value  chosen  is  arbitrary;  why  pick  50%? 

2.  Because  pd  can  increase  as  well  as  decrease  with  range,  there  may  be  more  than  one 
range  at  which  its  value  is  50%.  Figure  1  was  chosen  as  a  severe  example  of  this:  it  shows 
a  case  of  shallow  water  with  the  acoustic  energy  bouncing  between  bottom  and  surface 
several  times.  Which  of  the  range  values  at  which  pd  equals  50%  should  be  chosen  as  the 
detection  range? 

3.  The  quantity  pd  is  a  single-ping  value,  whereas  almost  always  sonar  systems  ping 
repeatedly.  This  operational  reality  should  be  taken  into  account,  not  only  for  its  own 
sake,  but  also  because,  when  comparing  different  sonar  systems,  using  a  metric  based  on 
pd  may  result  in  biased  decisions  if  the  different  systems  have  different  ping  rates  or 
patterns  of  transmission  (e.g.  omni-directional  compared  with  sector  compared  with 
ripple-fire(b)  etc.) 

4.  It  is  rarely  the  case  that  a  single  detection,  in  the  engineering  sense  of  the  term,  is 
sufficient  for  an  operator  to  declare  a  detection,  because  of  false  alarms.  Although  a 
specified  false-alarm  rate  is  part  of  the  input  to  calculations  like  those  shown  in  Figure  1, 
the  use  of  pd  alone,  without  explicit  reference  to  false-alarm  rate,  ignores  the  operational 


(b)'Ripple  fire'  refers  to  a  sonar  mode  in  which  transmissions  are  sent  out  sequentially  in  two  or  more 
directions  prior  to  listening.  The  rationale  is  that  this  puts  more  energy  into  the  water  in  a  given 
direction  than  would  be  possible  with  a  single  transmission  spread  simultaneously  over  all  the 
directions  of  interest. 
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impact  of  false  detections  that  are  difficult  to  distinguish  from  real  detections  and  that  are 
typically  produced  at  the  rate  of  several  per  ping. 

5.  If  more  than  one  sonar  system  is  available  and  operating,  then  ASW  effectiveness  can  be 
increased  through  appropriate  integration  of  detection  information  [3].  This  is  difficult  to 
assess  on  the  basis  of  single-ping  pd  alone. 

Points  1-3  have  long  been  acknowledged;  points  4  and  5,  though  probably  understood,  have 
rarely  been  taken  into  account  in  past  work.  Points  2  and  3  are  standardly  addressed  by 
moving  from  single-ping  Pj  values  to  a  cumulative  detection  probability  P  as  described  in 
the  next  subsection. 


2.2  Cumulative  Detection  Probability 

The  cumulative  detection  probability  P  q  is  the  probability  of  obtaining  a  target  detection  after 
a  series  of  pings. (c)  Its  calculation  involves  combining  several  p<j  values,  so  a  scheme  for 
selecting  the  actual  values  to  be  combined  is  required.  In  other  words,  a  scenario  must  be 
specified.  To  illustrate  how  this  works,  a  very  simple  scenario  is  introduced  in  §2.2.1. 
Subsections  2.2.2  and  2.2.3  detail  its  use  to  compute  P  j  values  and,  from  there,  detection 
range. 

2.2.1  Example  Scenario 

Figure  2  shows  schematically  a  simple  scenario  involving  a  surface  ship  transiting  at  constant 
heading  and  speed.  A  threat  submarine  ahead  of  the  ship  detects  it  at  bearing  b  (as  measured 
from  the  ship)  and  range  R ,  and  then  tracks  toward  intercept.  The  submarine  is  assumed  to 
transit  at  half  the  speed  of  the  surface  ship  and  to  set  a  course  so  that  bearing  b  remains 
constant  as  the  two  vessels  close.  A  surface-ship  speed  \/s  of  15  kn  was  chosen,  and  therefore 
the  submarine  speed  \/u  is  7.5  kn.  The  bearing  b  was  chosen  to  be  20°.  W 

For  simplicity,  it  is  assumed  that  the  surface  ship  pings  regularly  during  the  scenario,  using 
just  one  sonar  mode.(e)  The  scenario  should  start  at  a  range  R  in;tiaI  where  the  single-ping  pd 
values  are  negligibly  small. (f)  That  is,  Pj  values  as  a  function  of  R  are  required  as  inputs  to  the 
calculation.  The  scenario  can  be  stopped  at  any  convenient  value  of  R ;  in  the  examples 
presented  below,  we  stopped  at  R  =  1  km. 


(c)  In  this  note,  we  consistently  use  lower  case  p  to  denote  single-ping,  or  few-ping,  or  'instantaneous'  or 
'local'  probabilities  and  upper  case  P  to  denote  cumulative  probabilities.  (We  did  not  follow  this 
convention  in  earlier  publications,  e.g.  [3].) 

(d) The  maximum  possible  bearing  for  the  speeds  chosen  is  30°.  That  is,  for  b  >  30°,  intercept  is  possible 
only  if  the  submarine  increases  its  speed. 

(e)  More  complicated  search  schemes  —  for  example,  sector  sweeps  at  a  regular  sequence  of  bearings,  or 
some  initial  short  pings  to  clear  the  blind  zone  —  can  be  readily  incorporated  at  no  more  cost  than  the 
additional  book-keeping  required. 

(0  Uncritical  adherence  to  this  requirement  can  lead  to  a  difficulty  with  excessively  long  ping  intervals, 
as  discussed  in  the  next  subsection. 
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Figure2:  Schematic  diagram  ofthescenario  used  in  the  calculation  of  cumulative  detection  probabilities 

2.2.2  Computation  of  P  d 

The  cumulative  detection  probability  P  after  K  pings  is  given  by 

K 


(1) 


k= 1 


where  Pj  ^  is  the  single-ping  detection  probability  for  ping  k.  In  practice,  it  is  efficient  to 
compute  Pd  k  using  the  equivalent  recurrence  relation: 


(for  K  >  1). 


(2) 


With  the  initial  value  of  P^i  =  Pdi-  Ping  1  is  the  first  ping  following  scenario  start  at  range 
R  initial-  At  each  subsequent  ping  thereafter,  Pj  accumulates  according  to  the  recurrence 
relation.  Results  for  two  p<j  versus  range  curves  are  shown  in  Figure  3. 

The  range  at  which  to  start  the  scenario  can  be  a  significant  modelling  issue.  Some 
environments  show  significant  levels  of  instantaneous  detection  probability  at  large  distances, 
for  example,  when  a  convergence  zone  is  present.  However,  the  low  speed  of  sound  in  water 
means  that  long  look  distances  require  correspondingly  long  inter-ping  intervals.  For 
example,  the  out-and-back  acoustic  propagation  time  is  27  s  over  20  km,  40  s  over  30  km  and 
108  s  over  80  km,  using  a  speed  of  sound  in  water  of  1500  m  s'h  If  the  scenario  is  started  at 
80  km  range,  then  the  maximum  ping  rate  is  one  quarter  of  that  if  the  scenario  is  started  at 


4 


DSTO-TN-0932 


Figure  3:  Values  of  single-ping  detection  probability  pd  (full  black  line)  and  cumulative  detection  probability 

P  d(dotted  blueline)  for  two  environments  and  target  depths.  A  cumulation  ofPdbegan  atrangesof 
(a)  50  km  and  (b)  20  km.  The  range  value  at  each  blue  dot  shows  the  range  applying  at  the 
corresponding  ping. 

20  km  range.  This  can  cause  P  ^  to  accumulate  slowly.  (s)  Clearly,  there  is  a  trade-off  between 
ping  rate  and  utilising  whatever  instantaneous  Pj  may  be  available  at  large  range  [4],  This 
issue  should  be  addressed  on  a  case-by-case  basis  in  each  environment  where  it  arises,  having 
regard,  among  other  things,  to  likely  operational  practice  with  the  sonar  system  being 
modelled.(h) 

2.2.3  Detection  Range  from  Pd 

If  it  is  desired  to  quote  a  detection  range,  then  once  again  one  must  select  a  probability  value 
for  the  purpose.  As  Figure  3  illustrates,  cumulative  P  L\  often  rises  rapidly,  reaching  values 
close  to  1.0  at  quite  large  ranges.  For  this  reason,  50%  P  ^  can  seem  too  low  a  value  for  the 
definition  of  detection  range.  In  the  cases  shown  in  Figure  3,  one  would  be  tempted  to  choose 
a  P d  value  over  90%  so  that  the  resulting  detection  range  may  be  judged  to  be  'realistic'. 

2.2.4  Critique  and  Issues 

Cumulative  probabilities  are  monotonic;  that  is,  they  do  not  decrease  with  the  passage  of  time. 
This  eliminates  issue  2  on  p.  2:  choosing  a  P  ^  value  defines  a  unique  detection  range.  Issue  3  is 
also  dealt  with  in  principle,  since  any  characteristic  of  a  given  sonar  system— such  as  ripple- 
fire  modes  (footnote  b,  p.  2),  restrictions  on  ping  rates,  the  use  of  sector  searches  or  short 
pulses  to  clear  a  blind  zone,  the  effect  of  employment  tactics  with  dipping  sonars,  etc.  —  can  be 
included  as  part  of  the  scenario,  so  affecting  the  rate  at  which  P  ^  accumulates. 

Issue  5  (p.  3)  can  be  handled  in  principle  too,  by  extending  the  definition  of  cumulative 
detection  probability.  If  detections  from  two  sonar  systems  are  being  combined  then. 


fe)ln  principle,  this  limitation  could  be  overcome  by  coding  pings.  However,  it  would  also  be  necessary 
to  stagger  inter-ping  spacings  in  order  to  search  the  ring-shaped  blanking  zone  arising  from  the  fact 
that  monostatic  sonars  cannot  listen  while  transmitting.  Fielded  monostatic  sonar  systems  typically 
do  not  have  the  capability  of  interleaving  pings. 

(h)For  example,  if  the  environment  has  a  convergence  zone,  would  operators  seek  to  exploit  it? 


5 


DSTO-TN-0932 


assuming  that  the  sonars  are  acting  independently,  the  overall  cumulative  detection 
probability  of  the  suite  of  sonar  systems  is 


P 


d,  sonar  suite 


=  '-( 


1  ~P 


d,  sonar  1 


)( 


1  ~P 


d,  sonar  2 


)■ 


(3) 


with  an  obvious  generalisation  if  there  are  more  than  two  sonar  systems  contributing  to  the 
detector  suite. 


On  the  other  hand,  issue  1  on  p.  2  remains:  one  must  still  choose  a  P  d  value  at  which  to  declare 
a  'detection  range',  and  any  such  choice  is  necessarily  arbitrary.  If  anything,  the  issue  is  more 
acute  with  than  pd  because  the  'natural'  choice  of  50%  almost  always  gives  unrealistically 
large  detection  ranges  when  applied  to  Pd. 

Finally,  issue  4  on  p.  2  has  been  ignored.  Typical  false-alarm  rates  from  sonar  systems  mean 
that  the  detection  of  a  distant  submarine  might  not  be  recognised  among  the  clutter.  It  is 
usually  necessary  to  detect  the  submarine  more  than  once  in  order  to  distinguish  real  from 
false  detections.  This  consideration  exposes  the  problem  with  Pd  K,  which  gives  the 
cumulative  probability  of  just  a  single  detection  from  any  of  the  K  pings.  This  is  why  P^k  rises 
so  rapidly,  and  is  also  the  reason  why  automated  tracking  algorithms  typically  require,  say, 
3  detections  in  5  consecutive  pings  before  a  track  is  initiated. 

The  way  forward  is  now  clear:  we  propose  carrying  the  analysis  a  little  further  along  the  ASW 
kill  chain  to  compute  probabilities  connected  with  track  initiation,  as  described  in  the  next 
section. 


3.  Probability  of  Track  Initiation 

The  discussion  of  §2  shows  the  advantages  of  using  a  cumulative  probability.  The  most 
conceptually  straightforward  way  of  constructing  this  is  through  an  'instantaneous'  or  'local' 
track-initiation  probability.  This  is  described  in  §3.1.  The  cumulative  track-initiation 
probability  could  in  principle  be  computed  from  this,  but  it  turns  out  to  be  much  easier  to 
calculate  it  directly  from  detection  performance  using  a  Monte-Carlo  method,  as  described  in 
§3.2.2.  Subsection  3.2.3  gives  some  examples  comparing  the  behaviour  of  the  two  cumulative 
probabilities  —  of  detection  and  of  track  initiation. 

3.1  'Local' Track-Initiation  Probability 

Automated  tracking  systems  typically  apply  a  rule  such  as:  'start  tracking  an  object  if  it  is 
detected  m  or  more  times  in  n  opportunities'.  For  a  given  value  of  m,  the  'local'  track-initiation 
probability  pti  is: 

the  probability  that  a  track  is  initiated  after  exactly  n  pings. 

This  is  sometimes  called  an  'instantaneous'  probability,  but  the  term  is  misleading  because 
the  probability  refers  to  the  period  of  time  over  which  the  n  pings  are  emitted,  which  is 
typically  many  minutes.  Although  this  length  of  time  is  usually  small  compared  to  the 
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duration  of  a  scenario,  we  prefer  to  describe  the  probability  as  'local',  which  we  mean  in  a 
temporal  sense. 

Use  of  pti  is  common  in  studies  of  radar  (e.g.  [5]  and  refs,  therein).  For  sonar,  common  values 
of  m  and  n  are  3-in-5,  which  are  adopted  here  for  definiteness. (')  If  all  five  pings  have  the  same 
detection  probability  pd  and  are  statistically  independent,  then  a  simple  expression  for  pti 
results  [3]: 

Pti=pd3(l0-15pd+6pd2).  (4) 

This  expression  forms  the  basis  for  a  coverage-area  analysis  [3],  but  it  is  not  useful  in  the 
present  context  for  two  reasons.  First,  it  is  most  unlikely  in  any  realistic  scenario  that  pd  would 
be  the  same  for  all  five  pings.  It  is  possible,  using  the  rules  for  combining  probabilities,  to 
write  an  expression  like  Equation  (4)  for  a  situation  where  the  pd  ^  values  (1  <  k  <  5)  are  all 
different,  but  the  result  is  too  complicated  to  be  useful  and  much  too  complicated  to 
generalise  to  other  values  of  m  and  n . 

The  second  problem  with  Equation  (4)  concerns  a  significant  aspect  of  data  processing  in  real 
sonar  systems,  the  'data-association  problem'.  It  is  assumed  in  deriving  Equation  (4)  that 
every  detection  is  known  with  certainty  to  be  a  detection  of  the  target  of  interest.  Real-world 
ASW  sonar  systems,  on  the  other  hand,  typically  return  several  false  detections  with  every 
ping.  This  is  because  operators  almost  always  need  to  detect  submarines  at  ranges 
approaching  the  maximum  possible,  which  requires  them  to  set  the  detection  threshold  to  a 
low  value.  Many  false  detections  arising  from  random  noise  spikes  exceeding  the  threshold  is 
the  inevitable  consequence.  Since  a  false  detection  can  look  just  the  same  as  a  real  detection, 
which  of  the  detections  to  associate  with  the  target  can  be  a  significant  problem.  We  are 
currently  exploring  the  effect  of  false  detections  on  tracking,  including  track  initiation  [6,7]. 
For  present  purposes,  the  data-association  problem  can  be  resolved  in  one  of  two  ways: 

•  Perhaps  false  detections  are  not,  in  fact,  identical  to  real  detections.  They  might 
display  some  characteristic  that  allows  an  operator  to  distinguish  false  from  real,  such 
as,  for  example,  the  lack  of  Doppler  shift  or  a  distinctive  tonal  quality  in  the  returning 
echo. 

•  If  not,  then  the  problem  can  be  circumvented,  though  only  formally,  by  re-interpreting 
pd  to  mean  the  probability  of  detection  and  correct  data  association  (as  with,  for 
example,  the  use  of  the  product  PDPG  in  Reference  8).  The  difficulty  here  is  that 
acoustic-propagation  calculations,  which  provide  values  of  detection  probability  like 
those  shown  in  Figure  1(a),  do  not  provide  estimates  of  the  data-association 
probability. 

Another  complication  arises  from  the  fact  that  usually  many  more  than  n  pings  are  emitted 
during  a  scenario/))  We  deal  with  this  by  adopting  the  sliding-window  approach  illustrated  in 
Figure  4.  Groups  of  5  consecutive  pings  are  considered.  Each  group  has  a  pti  value  depending 
on  the  pd  ^  values  for  the  5  pings  concerned.  With  this  further  consideration,  there  would 


(i)  Examples  of  the  effect  of  choosing  other  values  are  given  in  ref.  [3]. 

(j)  Exceptions  to  this  may  occur  with  sonobuoys  or  dipping  sonars.  For  example,  a  dipper  may  be 
pinged  only  a  few  times  before  relocation.  Examples  of  this  sort  should  be  treated  case  by  case  on 
their  merits. 
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pings  1  2  3  4  5  6  7  8  9  - 

I  I  I  I  I  I  I  I  I 

, _ ,  position  1 

i _ i  position  2 

sliding  window  1 - 1  position  3 

i _ i  position  4 

i _ i  position  5 

Figure  4:  Sliding-window  concept  for  defining  a  'local'  track-initiation  probability  using  the3-in-5  rule  when 

more  than  5  pings  have  been  emitted.  T he  vertical  lines  represent  the  pings  and  the  long  brackets 
represent  successive  applications  of  a  sliding  window.  The  track  initiation  rule  is  defined  to  be 
satisfied  with  the  first  window  that  contains  3  detections. 

appear  to  be  little  point  in  pursuing  the  goal  of  writing  down  a  general  algebraic  expression 
for  pti  j the  track-initiation  probability  for  the  sliding  window  in  position  j.  Another  approach 
must  be  sought.  Our  suggestion  is  presented  in  §3.2.2,  following  the  definition  of  cumulative 
track-initiation  probability. 


3.2  Cumulative  Track-Initiation  Probability 

3.2.1  Definition 

The  cumulative  track-initiation  probability  Pti  is: 

the  probability  that  a  track  is  initiated  on  the  current  ping,  or  has  already  been  initiated  on  a 
previous  ping. 


If  expressions  were  available  for  ptij,  the  local  track-initiation  probability  for  the  sliding 
window  in  position  j,  then  these  could  be  used  to  compute  Ptij,  the  cumulative  track-initiation 
probability  after  J  positions  of  the  sliding  window  had  been  considered,  using  an  analogous 
expression  to  Equation  (1).  However,  values  of  ptij  are  not  available  for  reasons  discussed  in 
§3.1,(k)  so  we  proceed  as  described  in  the  next  subsection. 

3.2.2  Monte-Carlo  Calculation 

Cumulative  Pti  can  be  calculated  by  a  Monte-Carlo  method.  A  scenario  such  as  that  described 
in  §2.2.1  is  run,  say,  1000  times.  Each  run  has  exactly  the  same  number  of  pings  and  any  given 
ping  in  a  run  occurs  at  exactly  the  same  range  as  corresponding  pings  in  other  runs.  At  each 
ping  in  each  run,  a  random  number  is  drawn  from  a  uniform  distribution  on  the  interval  [0, 1] 
and  compared  with  the  local  pd  value  for  the  appropriate  range  to  determine  whether  that 
ping  resulted  in  a  detection.  In  this  manner,  each  run  produces  a  list  of  pings  marked  as  either 
detections  or  misses.  Figure  5  shows  schematically  part  of  the  set  of  runs  for  a  particular  sonar 
system,  environment,  target  type  and  target  depth,  with  ticks  marking  detections  and  crosses 
misses. 


(DCastella  [9]  gives  recurrence  relations  for  both  ptiJ  and  Pti  for  all  cases  with  n  <  4  but,  not  only  are 
these  difficult  to  generalise  to  larger  values  of  n,  also  he  assumes  equal  pd  for  all  pings.  Williams  [10] 
considers  varying  pd,  but  some  of  his  expressions  appear  to  be  erroneous  (in  particular  his 
Equations  2  and  3). 
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Figure5:  Schematic  representation  oftheresultofaM  onte-Carlocalculation  for  agiven  sonar,  environment, 
target  type  and  target  depth.  Parts  of  8  of  the  1000  runs  areshown.  Each  run  has  exactly  thesame 
number  of  pings,  with  detections  being  determined  randomly  according  to  the  instantaneous  pd 
vaiueapplying  at  each  ping.  T  icks  represent  detections  and  crosses  misses.  R  ed  arrows  show  pings  at 
which  the3-in-5  track  initiation  rule  is  first  satisfied,  assuming  that  it  was  not  satisfied  prior  to  ping 
18.  R  un  484  has  not  yet  achieved  track  initiation. 

Once  the  1000  runs  are  computed,  they  are  scanned  for  track  initiation.  That  is,  the  sliding 
window  is  applied  and  the  ping  number  at  which  track  initiation  is  first  achieved  is  marked  as 
shown  schematically  by  the  red  arrows  in  Figure  5. 

To  determine  Ptij  after  a  given  ping J,  the  number  of  runs  that  have  achieved  track  initiation 
on  or  before  that  ping  is  determined.  That  is,  one  counts  red  arrows  down  column  J  in 
Figure  5  and  adds  to  this  the  total  number  of  red  arrows  in  all  columns  to  the  left  of  column  J . 
This  number  expressed  as  a  fraction  of  1000  is  an  estimate  of  the  cumulative  Pti  following 
pingj.  The  accuracy  of  the  estimate  is  related  to  the  number  of  runs  and  can  be  explored  by 
computing  several  sets  of  1000  runs. 

3.2.3  Examples  of  Pd  and  Pti 

Figure  6  shows  four  examples  of  values  of  pd  (black  lines)/1)  Pd  (blue  dots)  and  Pti  (red 
crosses).  Each  dot  or  cross  corresponds  to  a  ping  in  the  scenario.  The  accumulation  begins  at 
the  scenario  start  range,  which  in  each  case  is  the  maximum  range  shown  on  the  abscissa.  The 
scenario  involves  closing  platforms,  so  range  becomes  smaller  as  time  progresses.  As  can  be 
seen,  Pd  reaches  quite  high  values  very  early  in  each  scenario,  at  relatively  large  ranges. 

The  rapid  increase  in  P  d  as  the  scenario  progresses  arises,  of  course,  because  the  submarine 
need  be  detected  just  once  to  fulfil  the  success  criterion  for  P  d.  Requiring  instead  3  detections 
in  5  consecutive  pings  overcomes  this  problem,  as  the  examples  in  Figure  6  show. 


I1)  The  pd  values  shown  do  not  take  account  of  the  blind  zone  near  zero  range,  the  extent  of  which 
depends  on  the  pulse  length  transmitted.  This  may  affect  the  accumulation  of  P  ti  near  the  end  of  the 
scenario;  in  particular,  the  rise  in  P  ti  values  apparent  at  short  range  in  panels  (c)  and  (d)  will  not 
occur  when  long  pulses  are  used. 


9 


DSTO-TN-0932 


Figure6:  Curves  of  instantaneous  detection  probability  pd  (black  lines),  cumulative  detection  probability  Pd 
(blue dots)  and  cumulativetrack-initiation  probability  Ptj  (red  crosses)  for  a  target  submarineat  the 
nominated  depths  in  the  nominated  environments  with  a  'typical'  modern  sonar  system.  In  each 
case,  accumulation  of  the  cumulative  probabilities  begins  at  the  maximum  range  shown  on  the 
respective  abscissas. 

Figure  6  highlights  a  well  known  feature  of  cumulative  probabilities:  they  never  decrease  with 
the  passage  of  time.  Some  operators  find  this  characteristic  disturbing.  As  an  extreme 
example,  the  P  L\  and  Pti  values  attained  in  Figure  6(d)  while  passing  over  the  convergence 
zone  beyond  60  km  range  are  maintained  through  the  succeeding  long  region  of  very  low  Pj 
values.  Similarly,  Pti  in  Figure  6(c)  plateaus  after  the  bottom  bounce  opportunity  around 
18  km  although  pt)  falls  to  quite  low  values  below  10  km.  Clearly,  these  cumulative  probability 
values  will  reflect  real  system  performance  only  if  the  sonar-system  operators  have  sufficient 
environmental  awareness  to  know  about  the  convergence-zone  and  bottom-bounce 
opportunities.  For  example,  detections  will  not  be  made  if  the  operators  have  selected  a  range 
scale  that  does  not  extend  out  as  far  as  the  range  of  the  convergence  zone  or  bottom  bounce. 

As  its  name  indicates,  Pti  concerns  track  initiation  only;  issues  of  track  maintenance  and  the 
increase  of  information  staleness  with  time  are  not  included.  Viewed  another  way,  the  use  of 
Pti  assesses  sonar  performance  in  search.  Once  a  track  is  initiated,  the  operator  may  switch  to 
an  'attack'  or  'investigate'  mode  (if  available),  thereby  concentrating  acoustic  energy  in  the 
direction  of  the  target.  This  would  mean,  among  other  things,  an  alteration  in  the  Pj  values. 


Another  issue  concerning  the  Pti  values  shown  in  Figure  6  is  the  effect  of  false  detections.  The 
calculation  resulting  in  the  Pj  curves  takes  a  false-detection  probability  as  an  input,  so  false 
detections  have  been  taken  into  account  at  that  level.  However,  in  the  real  world,  noise- 
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induced  false  detections  would  also  affect  track  initiation:  there  is  a  chance  that  a  false 
detection  could  occur  sufficiently  close  to  a  pair  of  real  detections  for  it  to  cause  a  track 
initiation  that  would  not  otherwise  have  occurred.  The  effect  is  not  included  in  the  method 
described  herein,  which  therefore  underestimates  Pti  by  an  amount  that  depends  on  the  false- 
detection  probability.  We  are  undertaking  a  more  elaborate  study  to  explore  this  [6,7], 


3.2.4  Integrating  Two  or  More  Sonar  Systems 


The  method  of  calculating  the  track-initiation  probability  for  a  sonar  suite  consisting  of  two  or 
more  sonar  systems  depends  on  the  manner  in  which  the  data  from  the  systems  are  fused. 
There  are  several  possibilities: 


Each  sonar  system  performs  its  own  tracking  and  does  not  pass  information  to  a 
central  location  until  a  track  is  initiated.  This  is  the  simplest  case.  For  systems  that 
act  independently  of  each  other  in  the  statistical  sense,  the  track-initiation  probability 
for  the  suite  is  given  by  an  expression  like  Equation  (3).  That  is,  for  a  suite  of  two 
sonars. 


p. 

1  ti,  sonar  suite 


1  (l  ^ti,sonarl)(l  ^h.sonarl)- 


(5) 


The  two  sonar  systems  need  not,  and  probably  do  not,  ping  at  the  same  time  or  with 
the  same  inter-ping  interval.  Equation  (5)  is  applied  each  time  either  sonar  pings  to 
update  the  Pti/  sonar  suite  value- 

•  The  sonar  systems  send  detection  information  to  a  centralised  tracker.  Depending  on 
environmental  details,  this  can  result  in  improved  performance  over  the  case  of 
individual  tracking,  as  shown  elsewhere  [3,11]. 

•  The  sonar  systems  operate  fully  multistatically.  That  is,  each  sonar  receives  and 
processes  echoes  not  only  from  its  own  pings,  but  also  from  those  of  other  sonars  in 
the  suite.  It  is  expected  that  this  would  result  in  a  further  improvement  in  perfor¬ 
mance,  but  the  analysis  of  multistatic  systems  is  complicated  and  is  not  considered 
here. 


Figure  7  shows  an  example  of  the  application  of  Equation  (5).  Here,  two  systems  with 
different  characteristics  share  track-level  information.  The  case  shown  in  Figure  7  is  extreme 
in  the  sense  that  neither  system  by  itself  achieves  a  Pti  of  0.5  until  quite  short  range,  but  the 
suite  reaches  that  value  well  out.  It  illustrates  the  advantage  that  can  accrue  even  with  the 
simplest  type  of  networking  and  in  the  smallest  of  networks  (i.e.  a  network  of  just  two 
systems).  It  also  shows  how  track-initiation  probability— whether  local  or  cumulative  — 
provides  a  way  of  quantifying  networking  effects,  which  single-ping  detection  probability 
does  not. 


3.3  Definition  of  'Detection  Range' 

Use  of  Pti  rather  than  overcomes  many  of  the  issues  listed  on  p.  2,  but  still  it  is  often 
desired  by  a  client  to  express  the  results  of  the  analysis  in  terms  of  detection  range.  When  this 
is  the  case,  there  seems  to  be  no  avoiding  the  arbitrary  choice  of  a  Pti  value  (issue  1  on  p.  2). 
Often,  a  value  of  50%  is  favoured,  with  inspection  of  plots  like  Figures  6  and  7  being  used  to 
support  the  'reality'  of  the  Pti  =  50%  rule  as  a  measure  of  detection  range.  In  our  experience. 
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Figure  7:  A  n  example  of  the  effect  of  track- level  fusion  of  data  from  two  sonar  systems  in  the  same  en  viron  - 
ment.  Theblack,  red  and  bluecurveshavethesamemeaningsas  in  Figure6,  thegreen  curvefwhich 
is  thesamein  both  panels)  gives  values  of  Pti  sonarsuite  Thebroken  lineshows  the 50%  probability 
value,  to  Indicate  the  range  at  which  Ptj  sonar  sujte  =  0.5. 

operators  and  other  clients  seem  to  feel  more  comfortable  with  the  50%  level  than  any  other.  It 
is  as  though  50%  is  regarded  as  more  natural  than  other  values.  Is  it,  perhaps,  that  the  use  of  a 
value  other  than  50%  draws  unwelcome  attention  to  the  arbitrariness  of  the  whole  process? 
This  touches  on  the  psychology  of  decision  making,  which  is  for  other  studies  to  address. 

Figure  6  also  provides  practical  support  for  a  detection-range  definition  using  the  50%  value 
in  that,  for  these  four  cases,  the  range  at  which  Pti  =  50%  is  little  different  from  that  at  which 
Pti  equals,  say,  90%.  This  is  because  the  Pti  curve  rises  so  sharply  in  the  relevant  region.  That 
is,  the  resulting  detection  range  is  insensitive  to  the  Pti  value  chosen,  within  reasonable  limits. 
This  feature  is,  however,  not  true  for  all  combinations  of  sonar  system,  environment,  time  of 
year,  target  type  and  depth.  As  an  extreme  example.  Figure  7  shows  a  case  where  the  value  of 
P  y,  Sonar  suite  hovers  only  a  little  above  50%  for  an  extended  range.  It  is  also  a  case  where 
forming  a  network  results  in  a  much  larger  detection  range  than  either  sonar  system  is  capable 
of  achieving  on  its  own. 

It  may  be  questioned  whether  a  range  based  on  Pti  should  be  termed  a  'detection  range'. 
Would  not  'track-initiation  range'  be  more  appropriate?  Possibly,  but  the  client  may  be 
unwilling  to  accept  an  unfamiliar  term.  We  can  rationalise  the  use  of  'detection  range'  in  terms 
of  the  difference  noted  in  §2.1  between  engineering  and  operator  interpretations  of  'detection'. 
A  strict  engineering  point  of  view  would  indeed  hold  that  the  detection  range  should  be 
defined  in  terms  of  P^  rather  than  Pti.  Flowever,  operators  understand  'detection'  to  include 
aspects  of  identification  and  the  filtering  out  of  false  detections.  Application  of  a  track- 
initiation  rule  involves  elements  of  the  second,  so  providing  a  level  of  justification  for  the 
usage. 
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4.  Summary  and  Conclusion 

In  this  note,  we  propose  the  use  of  cumulative  track-initiation  probability  Pti,  rather  than 
single-ping  or  cumulative  detection  probability,  as  a  basis  for  metrics  for  assessing  active- 
sonar-system  performance  in  anti-submarine  warfare.  The  definition  of  Pti  is:  the  probability 
that  a  track  is  initiated  on  the  current  ping,  or  has  already  been  initiated  on  a  previous  ping. 

The  issues,  advantages  and  disadvantages  of  Pti  compared  with  the  other  probabilities 
(including  local  pti)  are  summarised  schematically  in  Table  1.  In  detail: 

•  Cumulative  probabilities,  whether  Pti  or  Pd,  require  a  scenario  for  their  calculation. 
Hence  their  values  are  scenario  dependent  to  a  greater  extent  than  single-ping  or  local 
probabilities.  This  can  be  mitigated  somewhat  by  concentrating  on  scenarios  simple 
enough  to  form  basic  tactical  building  blocks  of  sufficient  generality  to  be  useful  to  a 
principal  warfare  officer  faced  with  the  task  of  formulating  a  detailed  tactical  response 
to  the  contingency  at  hand. 

•  Cumulative  probabilities  provide  a  basis  for  comparing  sonar  systems  with  different 
ping  rates  or  transmission  modes,  unlike  single-ping  or  local  probabilities. 

•  Single-ping  pd,  by  definition,  does  not  allow  the  analysis  of  the  effect  of  a  long  series  of 
pings,  unlike  the  cumulative  probabilities.  Local  pti  considers  groups  of  5  pings  only 
(or  n  pings  where  the  track-initiation  rule  is  m-in-n ). 

•  Track-initiation  probabilities  provide  some  recognition  of  the  problem  of  false 
detections,  unlike  detection  probabilities,  while  skirting  around  the  difficult  issue  of 
data  association. 

•  As  regards  several  sonar  systems  acting  together  and  pooling  data,  the  single-ping 
detection  probability  pd  provides  no  basis  for  analysis.  Combinations  of  several  pd 
values  are  required,  as  is  the  situation  for  the  other  three  probabilities.  However,  Pd 
permits  the  analysis  of  track-level  fusion  only,  whereas  pti  and  P  ti  allow  in  addition  the 
analysis  of  detection-level  data  fusion. 

•  Since  cumulative  probabilities  never  decrease  with  time,  they  provide  an  unambigu¬ 
ous  basis  for  defining  detection  range,  unlike  local  probabilities. 

•  The  actual  definition  of  detection  range  requires  the  choice  of  a  probability  value, 
which  is  unavoidably  arbitrary.  All  probabilities  have  this  deficiency,  but  it  is  worst 
for  cumulative  detection  probability  P  d,  because  the  'natural'  choice  of  50%  —  the  one 
value  that  an  operator  is  unlikely  to  query— gives  unrealistically  large  detection 
ranges  in  most  environments.  On  the  other  hand,  experience  suggests  that  Pti  =  50% 
works  well,  at  least  for  persistent  sonar  systems  like  ship-borne  hull-mounted  sonars 
and  towed  arrays. 

Finally,  we  note  that  the  purpose  of  any  piece  of  military  operations  analysis  is  to  inform  a 
decision  maker.  The  success  of  the  analysis  is  closely  related  to  how  useful  the  client  finds  it. 
Analytical  techniques  based  on  cumulative  track-initiation  probability  have  been  used  to 
inform  decisions  in  support  of  a  major  defence  acquisition  project,  to  the  satisfaction  of  the 
client. 
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Table  1:  Summary  of  strengths  and  weaknesses  of  various  probabilities  for  the  purposes  of  defining  the 
'detection  range'  of  a  sonar  system 


Probability: 

single-ping 

detection 

probability 

local 

track-initiation 

probability 

cumulative 

detection 

probability 

cumulative 

track-initiation 

probability 

independent  of  scenario* 

• 

• 

• 

• 

allows  comparison  of  sonars  with 
different  ping  rates  or  modes 

• 

• 

• 

• 

provides  some  recognition  of  the 
influence  of  false  detections 

• 

• 

• 

• 

includes  effect  of  many  repeated 
pings 

• 

• 

5  pings  only 

• 

• 

provides  a  basis  for  analysing 
networks  of  sonar  systems 

• 

• 

• 

track-level 
fusion  only 

• 

unambiguous  definition  of  detection 
range 

• 

• 

• 

• 

no  arbitrariness  in  definition 
of  detection  range 

• 

pd  =  50% 
acceptable 

• 

ptj  =  50% 
acceptable 

• 

Pd  =  50% 
unrealistic 

• 

Pti  =  50% 
acceptable 

*This  refers  to  the  disposition  and  motion  of  the  platforms  in  the  scenario.  All  probabilities  are  absolutely 
dependent  on  the  environment,  time  of  year,  nature  of  the  target  and  target  depth. 
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